Skip to content

chore: add dev container support for simplicity#581

Open
Tony-WLB wants to merge 2 commits intocanonical:mainfrom
Tony-WLB:feat
Open

chore: add dev container support for simplicity#581
Tony-WLB wants to merge 2 commits intocanonical:mainfrom
Tony-WLB:feat

Conversation

@Tony-WLB
Copy link
Copy Markdown
Contributor

This pull request introduces a development container setup to streamline onboarding and development for the Netplan project. It also completes the dependencies list.

Dev container environment:

  • .devcontainer/Dockerfile - define a reproducible Ubuntu 24.04-based development environment
  • .devcontainer/devcontainer.json - configure the dev container

Documentation updates:

  • Updated the README.md to mention the new dev container as an alternative to manual dependency installation, and revised the dependency list to match the packages installed in the dev container.

Comment thread .devcontainer/Dockerfile Outdated

WORKDIR /workspace

USER netplan
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try to use the default ubuntu user to avoid having to do useradd/usermod and stuff.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user is changed to the default ubuntu.

Comment thread .devcontainer/Dockerfile Outdated
python3-pytest \
python3-pytest-cov \
python3-yaml \
sudo \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really needed? Isn't it installed by default in ubuntu?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I double-checked that the sudo package is not included by default in the Docker Ubuntu image.

Comment thread .devcontainer/Dockerfile
Comment thread README.md Outdated

```sh
sudo apt install \
bash-completion \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned above, we can link to the Dockerfile here for the list of deps, to avoid them getting out of sync.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. This part refers to the # netplan dependencies section in the Dockerfile now.

Comment thread README.md Outdated

# Build dependencies

> If you prefer using a prebuilt development environment, see the dev container image definition in [`.devcontainer/Dockerfile`](.devcontainer/Dockerfile).
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a few commands for how you use Docker to start a container in your terminal. Something like this (completely untested):

docker build
docker run ./myfile
$ make
$ ./netplan foo  # or "make run"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build and run command examples have been added.

Comment thread README.md
gcovr \
pandoc
$ docker build -t netplan-dev -f .devcontainer/Dockerfile .
$ docker run -it --rm -v $(pwd):/workspace --name netplan-dev netplan-dev bash
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is helpful, and works on my machine. However, I think to make this useful we have to be able to build and run netplan from within the container. But make doesn't work for me:

ubuntu@c7be5a34e560:/workspace$ make
meson compile -C _build --verbose
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /workspace/_build -v
ninja: Entering directory `/workspace/_build'
ninja: error: '/home/ben.hoyt@canonical.com/w/netplan/features_h_generator.sh', needed by 'dbus/_features.h', missing and no known rule to make it
make: *** [Makefile:8: default] Error 1

Do you know why? I'm guessing it's an error with non-relative directories (the error references /home/ben.hoyt@canonical.com/w/netplan but I think that should be /workspace inside the container).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benhoyt I did not run into this issue. I am running this container build from within an Ubuntu VM and is compiling fine.

As you had mentioned, it is likely due to difference in the directory referred during the build.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's caused by the volume not being mounted correctly, -v $(pwd):/workspace. It combines the CWD with the workspace inside the container. Could you please try running this Docker container directly inside the netplan directory?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a make clean and that get rid of the first error (I guess it was leftover stuff in the _build directory from a non-Docker build). However, now I get this error with a clean container -- seems to be a permissions error creating anything inside the container:

$ docker run -it --rm -v $(pwd):/workspace --name netplan-dev netplan-dev bash
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@f41ed6926e5c:/workspace$ make
meson setup _build --prefix=/usr
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 357, in run
    app = MesonApp(options)
          ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 68, in __init__
    (self.source_dir, self.build_dir) = self.validate_dirs()
                                        ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 148, in validate_dirs
    (src_dir, build_dir) = self.validate_core_dirs(self.options.builddir, self.options.sourcedir)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 121, in validate_core_dirs
    os.makedirs(ndir1, exist_ok=True)
  File "<frozen os>", line 225, in makedirs
PermissionError: [Errno 13] Permission denied: '/workspace/_build'

ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
make: *** [Makefile:11: _build] Error 13
ubuntu@f41ed6926e5c:/workspace$ mkdir _build
mkdir: cannot create directory '_build': Permission denied
ubuntu@f41ed6926e5c:/workspace$ touch foo
touch: cannot touch 'foo': Permission denied

If you guys don't see the issue, I'll try to debug this with Jegadish tomorrow in our 1:1.

@JegadishD
Copy link
Copy Markdown

@Tony-WLB
The approach of using a devcontainer for netplan development looks promising.
However, the following points should be considered as part of using a container-based development environment.

  1. The netplan is used widely including in other debian based distributions. The original README file looks neutral and not leaning towards a specific container technology or IDE. As there are other container technologies and IDE, it is preferable to move the VSCode-container-build-environment details from the README.md to another VSC_specific file.

  2. Netplan has dependency with other processes like systemd, networkd, dbus. The container definitions do not include them. While running the make-tests, the following errors are seen which are likely due to lack of dependent services.

Failed to connect to system bus: No such file or directory
System has not been booted with systemd as init system (PID 1). Can't operate.
  1. Netplan code makes accesses to sys folder-files and kernel states depending on the network configurations. It needs to be seen if this limits us from running all tests from a container environment.

@Tony-WLB
Copy link
Copy Markdown
Contributor Author

Hi @JegadishD !

Yeah, I have the same concern about the VS Code part. Since editor choice is largely up to individual developers, I’m not sure how much emphasis we should put on VS Code here.
Would it make sense to move it to a separate file, or should we remove it altogether? VS Code already provides guidance through pop-ups when users open the project, so duplicating that here might not add much value.

The current dependencies part can cover the tests triggered by make check and make check-coverage. The networkd should be contained in the systemd package, and the dbus is included by the Ubuntu image.

However, as you mentioned, this container is not running systemd as PID 1, and that does bring some troubles for realistic testing. It seems that some manual setups are needed to run realistic tests with apply.
I've tried a naive test with netplan try in a VM instead of the container, and it was working. Maybe we can achieve similar behavior in the container with further configuration. For now, the container is okay withmake, make check, and make check-coverage; realistic tests may need a more detailed setup.

@JegadishD
Copy link
Copy Markdown

Hi @JegadishD !

Yeah, I have the same concern about the VS Code part. Since editor choice is largely up to individual developers, I’m not sure how much emphasis we should put on VS Code here. Would it make sense to move it to a separate file, or should we remove it altogether? VS Code already provides guidance through pop-ups when users open the project, so duplicating that here might not add much value.

Hi @Tony-WLB
Removing the VSCode dev environment details from this repository also makes sense. But it would be useful to have the Dockerfile, the devcontainer-file and the steps accessible at an another repository. We can make use (and extend) of it where applicable.

The current dependencies part can cover the tests triggered by make check and make check-coverage. The networkd should be contained in the systemd package, and the dbus is included by the Ubuntu image.

The systemd package is not present in the ubuntu docker container. In the process list of the container, I do not see systemd or networkd running in it.

Tasks:  20 total,   1 running,  19 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.4 us,  0.4 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
MiB Mem :   7935.5 total,   1588.4 free,   1782.2 used,   4875.7 buff/cache     
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   6153.3 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                 
    266 ubuntu    20   0   11.3g 137080  54784 S   1.4   1.7   0:14.20 node                                    
      1 ubuntu    20   0    2800   1748   1644 S   0.0   0.0   0:06.74 sh                                      
    138 ubuntu    20   0    2800   1960   1804 S   0.0   0.0   0:00.02 sh                                      
    228 ubuntu    20   0 1047520  54940  46544 S   0.0   0.7   0:00.20 node                                    
    257 ubuntu    20   0    2800   1916   1812 S   0.0   0.0   0:00.02 sh                                      
    598 ubuntu    20   0 1132448  73820  49164 S   0.0   0.9   0:09.07 node                                    
  36725 root      20   0    2800   1916   1812 S   0.0   0.0   0:00.01 sh                                      
  36731 root      20   0    2800   1768   1668 S   0.0   0.0   0:00.00 sh                                      
  36746 ubuntu    20   0    2800   1924   1812 S   0.0   0.0   0:00.01 sh                                      
  36797 root      20   0    2800   1940   1832 S   0.0   0.0   0:00.01 sh                                      
  36833 ubuntu    20   0    2800   1964   1808 S   0.0   0.0   0:00.01 sh                                      
  36961 ubuntu    20   0 1047376  55356  46308 S   0.0   0.7   0:00.04 node                                    
  36995 ubuntu    20   0  949652  55168  44400 S   0.0   0.7   0:00.11 node                                    
  37017 ubuntu    20   0  947516  53116  44400 S   0.0   0.7   0:00.11 node                                    
  37029 ubuntu    20   0 1132000  60228  49004 S   0.0   0.7   0:00.07 node                                    
  37042 ubuntu    20   0   52.9g 686868  62352 S   0.0   8.5   0:05.34 node                                    
  37073 ubuntu    20   0    5768   5232   3472 S   0.0   0.1   0:00.02 bash                                    
  37102 ubuntu    20   0 1026812  63980  48108 S   0.0   0.8   0:00.10 node                                    
  37246 ubuntu    20   0    8844   5388   3196 R   0.0   0.1   0:00.01 top                                     
  37270 ubuntu    20   0    2696   1692   1596 S   0.0   0.0   0:00.00 sleep  

However, as you mentioned, this container is not running systemd as PID 1, and that does bring some troubles for realistic testing. It seems that some manual setups are needed to run realistic tests with apply. I've tried a naive test with netplan try in a VM instead of the container, and it was working. Maybe we can achieve similar behavior in the container with further configuration. For now, the container is okay withmake, make check, and make check-coverage; realistic tests may need a more detailed setup.

Yes, The container dev environment can be be useful to an extent and the VM can be a suitable environment to test the real changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants